A Detailed Comparison of Two Transaction Processing Workloads

Commercial applications such as databases and Web servers constitute the most important market segment for high-performance servers. Among these applications, on-line transaction processing (OLTP) workloads provide a challenging set of requirements for system designs since they often exhibit inefficient executions dominated by a large memory stall component. A number of recent studies have characterized the behavior of transaction processing workloads and proposed architectural features to improve their performance. These studies have typically used a workload based on either the TPC-B or the TPC-C benchmark , with many of them opting for the simpler TPC-B benchmark. Given that the TPC-B and TPC-C workloads exhibit dramatically different characteristics on certain architectural metrics (such as cycles-per-instruction), it becomes important to find out whether the results or conclusions of these previous studies are heavily biased due to their choice of workload. This paper presents a detailed comparison of the debit-credit (modeled after TPC-B) and order-entry (modeled after TPC-C) transaction processing workloads in the context of various architectural choices. Our experiments use the Oracle commercial database engine for running the workloads, with results generated using both full system simulations and actual runs on Alpha multiprocessors. Our results confirm that certain characteristics of these workloads, such as cycles-per-instruction (CPI) and dirty miss frequency, are indeed quite different. Nonetheless, it turns out that the overall impact of most architectural choices (e.g., out-of-order execution, on-chip integration of system modules, chip multiprocessing) are surprisingly similar for the two workloads. Furthermore, the above similarity between the two workloads is sometimes due to non-intuitive effects that would be difficult to predict without conducting the experiment with both workloads. The findings in this paper make it easier to compare results from studies that use one or the other workload. Overall, we observe that for a wide range of architectural decisions that we considered , using the simpler TPC-B workload leads to virtually the same conclusions as using the more complex TPC-C workload. Finally, we show that these same conclusions hold across two generations of the Oracle database engine.